Bias , Variance , and Error Correcting Output Codes forLocal Learners ?

This paper focuses on a bias variance decomposition analysis of a local learning algorithm, the nearest neighbor classiier, that has been extended with error correcting output codes. This extended algorithm often considerably reduces the 0-1 (i.e., classiication) error in comparison with nearest neighbor (Ricci & Aha, 1997). The analysis presented here reveals that this performance improvement is obtained by drastically reducing bias at the cost of increasing variance. We also show that, even in classiication problems with few classes (m5), extending the codeword length beyond the limit that assures column separation yields an error reduction. This error reduction is not only in the variance, which is due to the voting mechanism used for error-correcting output codes, but also in the bias. Abstract. This paper focuses on a bias variance decomposition analysis of a local learning algorithm, the nearest neighbor classiier, that has been extended with error correcting output codes. This extended algorithm often considerably reduces the 0-1 (i.e., classiication) error in comparison with nearest neighbor (Ricci & Aha, 1997). The analysis presented here reveals that this performance improvement is obtained by drastically reducing bias at the cost of increasing variance. We also show that, even in classiication problems with few classes (m5), extending the codeword length beyond the limit that assures column separation yields an error reduction. This error reduction is not only in the variance, which is due to the voting mechanism used for error-correcting output codes, but also in the bias.